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1  Statement  of  the  Problem  Studied 


Under  this  grant  we  have  studied  the  development  of  a  scientifically  sound 
basis  for  software  development  that  builds  on  widely  used  pragmatic  meth¬ 
ods  but  is  firmly  grounded  in  well-established  formal  domains  such  as  first- 
order  logic  and  automata  theory.  To  be  sufficiently  expressive  for  software 
systems,  the  work  has  focused  on  methods  applicable  to  infinite-state  sys¬ 
tems.  Traditionally  methods  for  infinite-state  systems  have  been  expensive, 
because  they  were  mainly  deductive  and  thus  required  guidance  by  users 
who  were  both  experts  in  the  application  domain  and  in  the  verification 
methodology. 

Our  research  has  been  directed  at  algorithmic-deductive  techniques  that 
separate  the  combinatorial  reasoning  from  reasoning  about  the  data.  These 
methods  often  limit  user  input  to  providing  abstract  system  models  and 
application-level  guidance,  making  the  interaction  more  natural  to  software 
developers.  Constructed  proofs  hide  low-level  details;  instead,  they  reason 
at  the  most  appropriate  level  of  abstraction  with  respect  to  the  properties  to 
be  proved.  This  characteristic  of  proofs  make  them  suitable  as  system  doc¬ 
umentation  that  can  evolve  with  the  system.  To  ensure  well-defined  seman¬ 
tics,  computational  models  were  developed  for  new  computing  paradigms, 
including  aspects  of  publish-subscribe  systems  and  middleware  design  pat¬ 
terns. 
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2  Summary  of  Results 

2.1  Static  Analysis 

We  have  made  significant  contributions  to  the  automatic  construction  of 
proofs  of  sequential  and  reactive  systems  by  developing  a  new  approach  to 
invariant  generation  and  program  termination  analysis. 

2.1.1  Invariant  Generation 

Generating  system  invariants  is  one  of  the  most  important  components  of 
any  verification  methodology.  In  addition,  invariants  provide  insight  into 
the  system.  Invariant  generation  has  been  studied  extensively  for  several 
decades.  The  traditional  approaches  have  relied  on  forward  propagation  in 
an  abstract  interpretation  framework.  Starting  from  the  initial  condition, 
symbolic  simulation  is  performed  in  an  abstract  domain  until  a  fixed  point 
is  reached.  The  fixed  point  is  the  most  precise  invariant  of  the  system  rep¬ 
resentable  in  the  abstract  domain  chosen.  For  the  most  popular  abstract 
domains,  however,  convergence  may  not  be  reached  in  a  finite  number  of 
steps,  and  one  has  to  resort  to  widening  to  force  convergence,  thereby  mak¬ 
ing  the  invariant  less  precise  and  potentially  trivial.  Despite  much  study, 
widening  has  remained  an  art  more  than  a  science,  with  little  control  of  the 
user  over  the  resulting  precision,  and  often  unpredictable  results  in  practice. 

The  approach  that  we  have  developed  addresses  these  problems  by  posing 
the  invariant-generating  problem  as  a  constraint-solving  problem  [CSS03]. 
The  conditions  for  an  expression  of  a  certain  type  to  be  an  invariant  are 
encoded  as  a  constraint  system  and  the  solutions  of  this  constraint  system 
represent  all  invariants  of  that  type.  This  constraint-based  approach  has 
several  advantages  over  the  traditional  approach,  especially  for  software  en¬ 
gineering  practice. 

Controlling  Complexity  Invariant  generation  is  inherently  a  hard  prob¬ 
lem  with  high  complexity.  With  our  method,  however,  the  user  can 
make  deliberate  choices  how  to  trade  off  precision  versus  complexity 
by  choosing  the  shape  of  the  target  invariant  and  strengthening  the 
conditions  on  the  properties  to  be  found.  In  the  traditional  approach 
user  control  essentially  ends  with  the  choice  of  the  abstract  domain. 
Also,  our  approach  allows  exploitation  of  additional  structure  in  the 
system  to  reduce  the  complexity  of  the  constraint  solving.  For  exam¬ 
ple  in  [SSM03]  we  showed  that  systems  presented  as  Petri  nets  gave 
rise  to  linear  constraint  systems  rather  than  nonlinear  ones. 
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New  Abstract  Domains  In  the  traditional  approach  invariant  generation 
was  studied  mostly  in  the  abstract  domain  of  linear  inequalities.  At¬ 
tempts  to  extend  it  to  other  domains  were  largely  unsuccessful.  With 
our  constraint-based  approach,  however,  target  template  invariants 
can  be  chosen  in  any  domain  that  allows  the  encoding  of  the  conditions 
in  a  (decidable)  constraint  system.  For  example,  we  have  succeeded  in 
generating  nonlinear  invariants  (polynomial  equalities)  [SSM04]  and 
are  currently  investigating  application  to  recursive  datatypes.  In  soft¬ 
ware  engineering  practice  this  approach  creates  the  opportunity  to 
develop  invariant  generating  methods  specialized  for  the  application 
and  its  specific  data  structures. 

New  Target  Properties  Traditional  invariant  generation  methods  are  by 
their  very  nature,  forward  symbolic  simulation,  limited  to  generating 
invariants.  In  the  constraint-based  approach,  any  property  that  can 
be  encoded  as  a  constraint  system  can  be  generated.  For  example,  we 
have  applied  exactly  the  same  techniques  to  the  generation  of  ranking 
functions,  described  below.  In  software  engineering  practice,  this  can 
easily  be  extended  to  application-specific  properties.  We  are  currently 
developing  methods  to  generate  verification  diagrams  automatically. 

Constraint  Solvers  as  Engines  The  main  bottleneck  in  the  constraint- 
based  approach  is  the  complexity  of  solving  the  constraints.  Constraint¬ 
solving,  however,  is  an  independent  and  very  active  area  of  research 
with  many  other  applications.  The  advantage  of  the  constraint-based 
approach  is  that  any  advances  in  constraint  solving  can  directly  be  ex¬ 
ploited  by  our  methods.  Stronger  constraint  solvers  translate  directly 
into  improved  precision  and  increased  scalability. 

2.1.2  Termination  Analysis 

Guaranteed  termination  of  program  loops  is  necessary  in  many  settings, 
such  as  embedded  systems  and  safety  critical  software.  Although  termina¬ 
tion  analysis  methods  have  been  studied  extensively  in  logic  and  functional 
programming  and  term  rewriting  systems,  termination  of  imperative  pro¬ 
grams  had,  until  recently,  received  little  attention. 

Over  the  last  four  years  we  have  developed  a  systematic,  constraint-based 
approach  towards  termination  analysis  of  imperative  programs  [CS01,  CS02, 
BMS05].  In  general,  termination  of  loops  is  proved  by  exhibiting  a  rank¬ 
ing  function,  that  is,  a  function  that  is  well-founded  and  decreasing  with 
each  pass  through  the  loop.  We  have  developed  methods  for  automatically 
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synthesizing  such  ranking  functions.  Like  for  invariant  generation,  the  condi¬ 
tions  for  expressions  of  a  certain  type  to  be  a  ranking  function  are  encoded 
as  a  constraint  system.  If  the  resulting  constraint  system  is  satisfiable,  a 
ranking  function  exists,  and  hence  the  loop  has  been  proved  to  terminate. 
Unlike  for  invariant  generation,  the  constraint  systems  generated  for  ranking 
functions  axe  linear,  and  hence  can  be  solved  very  efficiently.  Therefore  this 
method  scales  remarkably  well.  We  have  demonstrated  our  methods  on  tens 
of  thousands  of  lines  of  code,  with  analysis  times  on  the  order  of  seconds. 

2.1.3  Static  Analysis  Tools 

The  lack  of  integration  between  prototype  implementations  of  results  of  re¬ 
search  blocks  progress  toward  direct  application  of  formal  methods  research 
in  software  engineering  settings.  We  have  surveyed  a  host  of  tools,  exam¬ 
ining  how  their  integration  would  increase  their  power  and  benefit  future 
research  and  application  [BSSM04]. 

2.2  Dynamic  Analysis 

Although  static  analysis  methods  have  improved  considerably,  in  many  cases 
they  still  do  not  scale  to  large  software  projects.  A  practical  alternative 
is  then  to  do  dynamic  analysis,  that  is,  to  monitor  the  running  program. 
Another  situation  in  which  dynamic  analysis  may  be  the  only  option  is  when 
the  program  must  run  in  an  environment  that  does  not  tolerate  violations 
of  the  specification,  but  the  source  code  is  not  available  for  inspection  for 
proprietary  reasons  or  due  to  outsourcing. 

Dynamic  analysis  tends  to  have  lower  complexity  than  static  analysis, 
because  runs  of  the  system  are  analyzed  individually,  while  with  static  anal¬ 
ysis  all  (usually  infinitely  many)  possible  runs  must  be  covered.  We  have 
developed  efficient  methods  for  runtime  verification  based  on  alternating  au¬ 
tomata  [FS04,  FSS02] .  These  methods  are  not  limited  to  checking  temporal 
properties,  but  can  also  collect  runtime  statistics.  These  statistics  are  useful 
as  early  warnings  of  impending  problems  in  performance  and  resource  usage. 

Dynamic  analysis  methods  can  also  be  used  as  an  alternative  to  test¬ 
ing.  In  collaboration  with  Synopsys  we  have  developed  a  specification  lan¬ 
guage  and  algorithms  for  the  online  and  offline  monitoring  of  synchronous 
systems  including  circuits  and  embedded  systems  [DSS+05].  The  specifica¬ 
tion  language  can  describe  both  correctness/failure  assertions  and  statistical 
measures  that  are  useful  for  system  profiling  and  coverage  analysis. 
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2.3  Computational  Models 

New  computational  paradigms  require  well-defined  computational  models  as 
a  basis  for  reasoning  about  systems  developed  according  to  these  patterns 
and  principles.  Often,  new  paradigms  emerge  and  are  adopted  because  they 
are  found  to  solve  recurring  problems  conveniently.  Modeling  and  anal¬ 
ysis  come  afterward.  We  have  modeled  and  analyzed  the  following  new 
paradigms: 

Aspect-oriented  programming  Aspect-oriented  programming  [KLM+97] 
allows  component-based  development  with  orthogonal  concerns  such 
as  security  or  concurrency  control  developed  and  incorporated  sepa¬ 
rately.  We  developed  a  computational  model  for  aspect-oriented  con¬ 
struction  of  reactive  systems,  which  allows  analysis  of  preservation 
of  temporal  properties  [Sip03]  based  on  early  work  in  this  area  by 
Katz  [Kat93]. 

Publish-subscribe  Systems  The  publish-subscribe  paradigm  has  emerged 
as  a  convenient  architectural  principle  to  construct  large  loosely-coupled 
distributed  systems.  Components  publish  messages  to  the  middleware, 
which  distributes  them  to  components  that  have  expressed  interest  by 
means  of  subscriptions.  Subscriptions  can  be  in  the  form  of  simple 
filters  or  as  more  complex  temporal  patterns,  also  known  as  event  cor¬ 
relation  expressions.  In  safety  critical  systems,  but  also  in  many  other 
systems  such  as  stock-trading  systems,  it  is  essential  that  all  relevant 
messages  are  delivered  to  their  target  audiences.  Practical  experience 
with  a  popular,  open-source  middleware  platform  showed  that  it  is 
extremely  hard  to  get  the  semantics  of  an  event-correlation  service 
correct  without  a  very  careful  analysis. 

We  developed  an  event-correlation  language  and  defined  its  operational 
semantics  in  terms  of  concurrent  automata  [SSS+03].  This  semantics 
not  only  allows  analysis  of  event  correlation  expressions  [SSSM05],  but 
also  enables  automatic  synthesis  of  the  the  correlators  to  be  embedded 
in  the  middleware,  thus  obtaining  correct-by-construction  implemen¬ 
tations. 

Design  Patterns  Design  patterns  [GHJV95]  provide  schematic,  informal 
solutions  to  frequently  occurring  problems  in  software  development. 
Formalizing  such  patterns  is  an  important  first  step  in  enabling  for¬ 
mal  analysis  of  various  aspects  of  systems  developed  according  to  these 
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patterns.  Together  with  middleware  experts  from  Washington  Univer¬ 
sity  in  St  Louis,  we  formalized  one  such  design  pattern  for  middleware 
systems,  known  as  WaitonConnection  [SSRBOO].  We  modeled  remote 
invocations  handled  in  accordance  with  this  pattern  and  analyzed  re¬ 
source  requirements  and  potential  for  deadlock  for  different  thread 
allocation  protocols  [SSS+05].  This  formalization  can  serve  as  an  ex¬ 
ample  for  modeling  other  design  patterns. 

2.4  Decision  Procedures 

Efficient  decision  procedures  are  the  cornerstone  of  every  verification  sys¬ 
tem.  Decision  procedures  for  individual  theories,  however,  are  of  limited  use 
in  verification  and  especially  in  software  verification,  because  most  verifica¬ 
tion  conditions  involve  data  types  that  span  multiple  theories,  for  example 
recursive  data  types  combined  with  integers. 

We  have  developed  combination  methods  for  various  data  types  with  in¬ 
tegers,  including  recursive  data  types  and  queues  [ZSM04a,  ZSM04b,  ZSM05]. 

2.5  Case  Studies:  CARA 

Automation  of  medical  devices  can  save  lives.  It  can  assure  continuous  mon¬ 
itoring  and  control  and  consistent  care  when  trained  medical  personnel  is 
not  available  or  in  short  supply.  On  the  other  hand,  errors  in  the  software  or 
missed  conditions  can  be  fatal.  The  Food  and  Drug  Administration  (FDA) 
has  the  task  to  determine  whether  a  medical  device  is  safe  for  deployment. 
Currently  this  is  done  by  extensive  review  of  the  development  and  testing 
process;  no  formal  verification  is  mandated  to  obtain  approval. 

As  software  gets  more  and  more  complex  there  is  a  feeling  that  the 
current  review  process  may  not  be  adequate,  and  more  formal  techniques  are 
called  for.  The  CARA  case  study  was  initiated  by  the  FDA,  in  collaboration 
with  the  Army  Research  Office,  to  assess  the  feasibility  of  doing  formal 
verification  for  such  a  device 

The  CARA  case  study  concerns  a  computer-assisted  resuscitation  pump 
used  to  provide  fluids  to  people  who  suffer  severe  loss  of  blood,  to  stabilize 
their  blood  pressure  until  more  permanent  remedial  actions  such  as  surgery 
can  be  provided.  The  device  was  developed  by  the  Walter  Reid  Army  Insti¬ 
tute  of  Research  (WRAIR)  in  Silver  Spring,  MD. 

We  were  provided  with  a  set  of  tagged  requirements,  developed  by  med¬ 
ical  experts  at  WRAIR.  The  same  document  was  also  given  to  the  software 
engineers  charged  with  developing  the  software.  We  participated  in  several 
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meetings  and  rounds  of  questions  and  answers  with  the  medical  expert  and 
software  engineers  at  WRAIR. 

We  modeled  the  system  using  clocked  transition  systems  [MP95,  KMP96, 
KMP98],  an  extension  of  fair  transition  systems  to  account  for  continuous 
real  time.  In  the  course  of  modeling  this  system  we  added  several  constructs 
to  make  the  description  more  natural.  Clocked  transition  systems  are  a  very 
expressive  model,  and  thus  most  questions  about  them  are  undecidable. 
However,  the  model  under  construction  did  not  use  the  full  expressiveness, 
and  thus  we  were  able  to  specialize  our  analysis  techniques  for  this  case, 
up  to  the  point  where  model  checking  could  be  used  for  some  of  it.  The 
analysis  techniques  were  implemented  in  our  verification  tool  STeP  (Stanford 
Temporal  Prover)  [BBC+00]. 

2.6  Tool  Development:  STeP  +  AutoFocus 

We  have  collaborated  with  researchers  from  the  group  of  Prof.  Manfred 
Broy  of  the  Technical  University  of  Munich  on  the  integration  of  STeP  with 
AutoFocus.  Dr.  Heiko  Lotzbeyer  and  Alexander  Wisspeintner  from  the 
Technical  University  of  Munich  visited  SRI  and  Stanford  for  several  weeks. 
We  explored  the  integration  of  design  and  verification  models  between  Auto¬ 
Focus  [HSSS96]  and  STeP  [BBC+00].  We  translated  an  existing  AutoFocus 
model  into  STeP,  while  carefully  considering  both  the  structural  and  the 
behavioral  aspects  of  the  model.  Extra  variables  had  to  be  introduced  to 
preserve  the  structural  view,  consisting  of  a  network  of  components  con¬ 
nected  by  communication  channels,  and  its  execution  semantics.  The  be¬ 
havioral  view  could  directly  be  modeled  by  STeP  transition  systems,  with 
the  addition  of  a  global  clock  to  maintain  the  discrete  time  semantics  of 
AutoFocus. 
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